GENDIRECT: a GENeralized DIRECT-type algorithmic framework for derivative-free global optimization

Over the past three decades, numerous articles have been published discussing the renowned DIRECT algorithm (DIvididing RECTangles). These articles present innovative ideas to enhance its performance and adapt it to various types of optimization problems. A comprehensive collection of deterministic, derivative-free algorithmic implementations based on the DIRECT framework has recently been introduced as part of the DIRECTGO project. DIRECTGO empowers users to conveniently employ diverse DIRECT-type algorithms, enabling efficient solutions to practical optimization problems. Despite their variations, DIRECT-type algorithms share a common algorithmic structure and typically differ only at certain steps. Therefore, we propose GENDIRECT -- GENeralized DIRECT-type framework that encompasses and unifies DIRECT-type algorithms into a single, generalized framework within this paper. GENDIRECT offers a practical alternative to the creation of yet another ``new'' DIRECT-type algorithm that closely resembles existing ones. Instead, GENDIRECT allows the efficient generation of known or novel DIRECT-type optimization algorithms by assembling different algorithmic components. This approach provides considerably more flexibility compared to both the DIRECTGO toolbox and individual DIRECT-type algorithms. A few hundred thousand DIRECT-type algorithms can be combined using GENDIRECT, facilitating users' easy customization and the addition of new algorithmic components. By modifying specific components of five highly promising DIRECT-type algorithms found in the existing literature using GENDIRECT, the significant potential of GENDIRECT has been demonstrated. The resulting newly developed improved approaches exhibit greater efficiency and enhanced robustness in dealing with problems of varying complexity.Comment: 29 pages, 6 figures, 8 table

DIRECT algorithm for constrained global optimization problems

Magistro baigiamajame darbe išanalizuotos ir įvertinos modifikacijos su kuriomis DIRECT algoritmas geba spręsti problemas su įvairiais apribojimais. Sukurta dviejų fazių baudų parametrų generavimo strategija kuria su tikslia L1 baudos funkcija ir adaptuotu DIRECT algoritmu efektyviai sprendžia nagrinėtus optimizavimo uždavinius kuriuose leistinoji sritis yra ne stačiakampio formos. Pirmoje magistro darbo dalyje apžvelgiama teorinė DIRECT algoritmo bazė, reikalaujamos prielaidos bei pateikiama praktinė veikimo schema. Antroji dalis orientuota į modifikacijas skirtas uždaviniams su bendrojo tipo tiesiniais ir netiesiniais ribojimais bei egzistuojančių modifikacijų trūkumų atskleidimą ir žinomų modifikacijų tobulinimą. Šioje dalyje be jau paminėtos dviejų fazių baudų generavimo strategijos pasiūlytos ir dar dvi strategijos skirtos baudų parametrams parinkti, tačiau jos nusileidžia pirmajai. Galutinėje, trečiojoje dalyje, pateikiamos pasiūlytų modifikacijų realizacijos, bei praktiškai ištirta pasiūlytų modifikacijų efektyvumas.In this master's thesis analyzed and evaluated the modifications which DIRECT algorithm is able to solve problems with multiple constraints. Developed two-phase penalty parameters generation strategy for Exact L1 penalty function effectively solves the tasks when the feasible area is not a rectangular shape. In the first paragraph of master's thesis is analyzed the theoretical DIRECT algorithm working scheme. In the second part emphasis was on modifications for the problems with other types of constrains. Disclosure the weakness of modifications and possible modifications improvements. In this paragraph without already mentioned two-phase penalty parameters generation strategy also was developed and offered two more strategies for Exact L1 penalty function but the results of those strategies descend to the first. In the third part, practically investigated the possibilities of old and new modifications.Švietimo akademijaVytauto Didžiojo universiteta

Globaliojo optimizavimo algoritmų,nereikalaujančių išvestinių informacijos, kūrimas, tobulinimas ir realizacija

Due to its simplicity and efficiency, the derivative-free global-search DIRECT (DIviding RECTangles) algorithm has received much consideration from the optimization community, and various novel ideas and extensions have been proposed. However, the efficiency of many DIRECT-type algorithms solving multi-modal problems and also in cases when a solution with high accuracy is required has decreased. This thesis presents the new scheme for selecting potentially optimal hyper-rectangles in the DIRECT framework, which addresses two of its weaknesses. An extensive experimental investigation revealed the potential and competitiveness of the added enhancements in our recent proposals, especially for more challenging multi-modal optimization problems. Unfortunately, the original DIRECT algorithm addresses optimization problems only with bounds on the variables, and due to it, the application of the algorithm is limited, as various applied optimization problems often hold other types of constraints. The initial DIRECT extensions for problems with general constraints were not competitive, compared with other derivative-free global optimization methods. Only in recent years, a few promising DIRECT-type modifications were proposed. In this thesis, two different constraint handling techniques are presented, and one of these strategies can even be applied to solve problems with hidden constraints. The proposed algorithms effectively explore hyper-rectangles with infeasible midpoints close to the boundaries of feasibility and may cover feasible regions. An extensive experimental investigation revealed the potential of the proposed approaches compared with other existing DIRECT-type algorithms for constrained global optimization problems, including important engineering issues. Contemporary problems often can not be solved with algorithms reasonably fast using a single core on the fastest computers. However, most DIRECT-type algorithms present challenges for efficient parallel implementation, and only a few parallel versions of DIRECT are known. To the best of our knowledge, all the existing parallel DIRECT-type versions are focused on box-constrained global optimization problems. Since the newly proposed selection scheme per iteration selects a larger number of subdividing regions, the algorithms developed in this thesis look more promising for parallelization than DIRECT. Therefore, the first parallel DIRECT-type algorithms for constrained global optimization problems are also introduced in this thesis

Improvement, development and implementation of derivative-free global optimization algorithms

Due to its simplicity and efficiency, the derivative-free global-search DIRECT (DIviding RECTangles) algorithm has received much consideration from the optimization community, and various novel ideas and extensions have been proposed. However, the efficiency of many DIRECT-type algorithms solving multi-modal problems and also in cases when a solution with high accuracy is required has decreased. This thesis presents the new scheme for selecting potentially optimal hyper-rectangles in the DIRECT framework, which addresses two of its weaknesses. An extensive experimental investigation revealed the potential and competitiveness of the added enhancements in our recent proposals, especially for more challenging multi-modal optimization problems. Unfortunately, the original DIRECT algorithm addresses optimization problems only with bounds on the variables, and due to it, the application of the algorithm is limited, as various applied optimization problems often hold other types of constraints. The initial DIRECT extensions for problems with general constraints were not competitive, compared with other derivative-free global optimization methods. Only in recent years, a few promising DIRECT-type modifications were proposed. In this thesis, two different constraint handling techniques are presented, and one of these strategies can even be applied to solve problems with hidden constraints. The proposed algorithms effectively explore hyper-rectangles with infeasible midpoints close to the boundaries of feasibility and may cover feasible regions. An extensive experimental investigation revealed the potential of the proposed approaches compared with other existing DIRECT-type algorithms for constrained global optimization problems, including important engineering issues. Contemporary problems often can not be solved with algorithms reasonably fast using a single core on the fastest computers. However, most DIRECT-type algorithms present challenges for efficient parallel implementation, and only a few parallel versions of DIRECT are known. To the best of our knowledge, all the existing parallel DIRECT-type versions are focused on box-constrained global optimization problems. Since the newly proposed selection scheme per iteration selects a larger number of subdividing regions, the algorithms developed in this thesis look more promising for parallelization than DIRECT. Therefore, the first parallel DIRECT-type algorithms for constrained global optimization problems are also introduced in this thesis

Novel algorithm for linearly constrained derivative free global optimization of Lipschitz functions /

This paper introduces an innovative extension of the DIRECT algorithm specifically designed to solve global optimization problems that involve Lipschitz continuous functions subject to linear constraints. Our approach builds upon recent advancements in DIRECT-type algorithms, incorporating novel techniques for partitioning and selecting potential optimal hyper-rectangles. A key contribution lies in applying a new mapping technique to eliminate the infeasible region efficiently. This allows calculations to be performed only within the feasible region defined by linear constraints. We perform extensive tests using a diverse set of benchmark problems to evaluate the effectiveness and performance of the proposed algorithm compared to existing DIRECT solvers. Statistical analyses using Friedman and Wilcoxon tests demonstrate the superiority of a new algorithm in solving such problems

Experimental study of excessive local refinement reduction techniques for global optimization DIRECT-type algorithms

This article considers a box-constrained global optimization problem for Lipschitz continuous functions with an unknown Lipschitz constant. The well-known derivative-free global search algorithm DIRECT (DIvide RECTangle) is a promising approach for such problems. Several studies have shown that recent two-step (global and local) Pareto selection-based algorithms are very efficient among all DIRECT-type approaches. However, despite its encouraging performance, it was also observed that the candidate selection procedure has two possible shortcomings. First, there is no limit on how small the size of selected candidates can be. Secondly, a balancing strategy between global and local candidate selection is missing. Therefore, it may waste function evaluations by over-exploring the current local minimum and delaying finding the global one. This paper reviews and employs different strategies in a two-step Pareto selection framework (1-DTC-GL) to overcome these limitations. A detailed experimental study has revealed that existing strategies do not always improve and sometimes even worsen results. Since 1-DTC-GL is a DIRECT-type algorithm, the results of this paper provide general guidance for all DIRECT-type algorithms on how to deal with excessive local refinement more efficiently

Of "Google" drive for statistical research

Vytauto Didžiojo universitetasŠvietimo akademij

Experimental Study of Excessive Local Refinement Reduction Techniques for Global Optimization DIRECT-Type Algorithms

This article considers a box-constrained global optimization problem for Lipschitz continuous functions with an unknown Lipschitz constant. The well-known derivative-free global search algorithm DIRECT (DIvide RECTangle) is a promising approach for such problems. Several studies have shown that recent two-step (global and local) Pareto selection-based algorithms are very efficient among all DIRECT-type approaches. However, despite its encouraging performance, it was also observed that the candidate selection procedure has two possible shortcomings. First, there is no limit on how small the size of selected candidates can be. Secondly, a balancing strategy between global and local candidate selection is missing. Therefore, it may waste function evaluations by over-exploring the current local minimum and delaying finding the global one. This paper reviews and employs different strategies in a two-step Pareto selection framework (1-DTC-GL) to overcome these limitations. A detailed experimental study has revealed that existing strategies do not always improve and sometimes even worsen results. Since 1-DTC-GL is a DIRECT-type algorithm, the results of this paper provide general guidance for all DIRECT-type algorithms on how to deal with excessive local refinement more efficiently